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04 ■ Abstract. A consistent kernel estimator of the limiting spectral distri- 
bution of general sample covariance matrices was introduced in Jing, Pan, 

^ | 1 ■ Shao and Zhou (2010). The central limit theorem of the kernel estimator 

\ is proved in this paper. 
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c3 . 1. Introduction 



> 
^1- 



Spectral analysis of sample covariance matrices plays a very important role 
in multivariate statistical inference since many test statistics are defined by its 
eigenvalues or functionals. Let X = (Xij) pxn be independent and identically 
i/-) \ distributed (i.i.d.) real- valued random variables and T be a p x p non-random 

0\ ■ Hermitian non-negative definite matrix with (T 1 / 2 ) 2 = T. Define the sample 

covariance matrix by 

O . n 

and its empirical spectral distribution by 

> ■ p 

H ■ P ^ 



c3 



k=l 

where k = 1, • • • ,p denote the eigenvalues of A n . Instead of A n we also 
consider 

1 rp 

B n = — X n TX n , 

n 



1991 Mathematics Subject Classification. Primary 15B52, 60F15, 62E20; Secondary 
60F17. 

Key words and phrases, sample covariance matrices, Stieltjes transform, nonparametric 
estimate, central limit theorem. 

G.M. Pan was partially supported by a grant M58110052 at the Nanyang Technological 
University; Q.M. Shao was partially supported by Hong Kong RGC CERG 602608 and 
603710; W. Zhou was partially supported by a grant R-155-000-106-112 at the National 
University of Singapore. 

1 



2 



GUANGMING PAN, QI-MAN SHAO, WANG ZHOU 



because the eigenvalues of A n and B„ differ by \n—p\ zero eigenvalues. Suppose 
the ratio of the dimension and sample size c n = p/n tends to a positive constant 
c as n — > oo. When F T converges weakly to a distribution H, it is proved in 
Marcenko and Pastur [8], Yin [16] and Silverstein [12] that, with probability 
one, F Bn (x) converges in distribution to an MP type distribution function 
F^ ,H (x) whose Stieltjes transform m(z) = m F c,H(z) is, for each z G C + = {z G 
C : Qz > 0}, the unique solution to the equation 

rtdH(t)\~ l 

, 1.1 ) m — — z — c 



f tdH(t)\ 
J 1 + tm) 



Here the Stieltjes transform mp(z) for any probability distribution function 
F(x) is given by 

(1.2) m F (z) = I dF(x), zeC + . 



x — z 

Note that from ( 11.11) m(z) has an inverse 

1 ft 

(1.3) z = + c / dH(t). 

Bai and Silverstein [2] established a far reaching central limit theorem (CLT) 
for the eigenvalues of A n , which makes possible the hypothesis testing of linear 
spectral statistics of sample covariance matrices indexed by analytic functions. 
Pan and Zhou [9] relaxed some restriction on the fourth moment of the under- 
lying random variables. Lytova and Marcenko [7] and Bai, Wang and Zhou [3] 
further, respectively, extended Bai and Silverstein's theorem from the analytic 
test function to the one having fourth derivative when T is the identity ma- 
trix. However, the limiting spectral distribution F_ C,H is usually unknown for 
general T. It is also not clear if there is any CLT about (_F Bn (x) — F_ c,H (x)), 
equivalently ( even in the normal population, here F c ' (x) 

is the limiting distribution of F^ n (x). How can one make inference for f c ,H(x) 
or F c,H {x) based on F^ n (x) or F Bn (x) without establishing CLTs? 

Motivated by the "smoothing" ideas, Jing, Pan, Shao and Zhou [6] proposed 
the following kernel estimator of the density function of F_ c,H (x) as 

(1.4) /„(») = ±±K^) - I J KC-^W^y), 

where h is the bandwidth. It was proved that f n (x) is a consistent estimator 
of Jc,h(x) under some regularity conditions. 

The main aim of this paper is to establish a CLT for f n {x)- This provides 
an approach to making inference on the MP type distribution functions. To 
this end, we first list some technical conditions on the kernel function. 
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Suppose that the kernel function K(x) satisfies 
'1.5) lim |xK(x)| = lim Ixii'^a:)! = 0, 

\x\— >ao \x\—>oo 



(1.6) /*(,)* = !, / W x)|«fa<oo, /|**(x)|<b< 
and 

(1.7) / xK(x)dx = 0, / x 2 |fsT(a;)|da; < oo. 



oo. 



Let z = u + iv with v being in a bounded interval, say [-Do,fo] with t> > 0. 
Suppose that 



. +oo 

1.8) / \K u) (z)\du< oo, j = 0,1, 2, 



uniformly in f G [— fo,t>o], where K^\z) denotes the fc-th derivative of i£"(#). 

Some assumptions on H n (t) := F T , are also needed. Introduce the interval 

(1.9) [A min (T)(l - v/^) 2 , A max (T)(l + ^ n ) 2 ] . 

Denote the right and left end points of the above interval, respectively, by 
di and 02- We then introduce a contour C\ as the union of four segments 
1j,3 = 1,2,3,4. Here 

7i = u — iv h, u G [ai, Or], 72 = u + iv h, u G [at, a r ], 

73 = o-i + iv,v G [—v h,v h], 74 = a r + if ,f G [— t> /i, Vo^i], 
where a; is any positive value smaller than the left end point of (11. 9p . a r any 
value larger than the right end point of ( 11. 91) . and Vo is specified in ( 11. 8ft . 
Assume that on the contour Ci 

where ^(z) = t> > 0, Mi is a positive constant and m° (z) is the Stieltjes 
transform of the distribution function F_ Cn,Hn (x) which is obtained from F_ c,H (x) 
with c and H replaced by c n and H n . Also, on the contour C\ we assume that 

f dHJt) 

(1.11) / 1 " v XM < M 

and that 

(1 - 12) /ra*< M < 

where Em n (z) is the expectation of the Stieltjes transform of F Bn and M is a 
constant independent of n and 2. The main results are stated below. 
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Theorem 1. Suppose that 

1) h = h(n) is a sequence of positive constants satisfying 
(1.13) lim nh 5/2 = oo, lim nh 3 = 0, lim h = ; 

2) X(x) satisfies ( fI.5]) -( TOj] and zs analytic on open interval including 

a 2 — a\ ax — 02-1 

3) are iid. wift EX U = 0, Var(Xu) = 1, ^X^ = 3 and EX^ 6 < 
oo, c n — )■ c G (0, 1) ; 

4) T zs a p x p non-random symmetric positive definite matrix with spec- 
tral norm bounded above by a positive constant such that H n = F T 
converges weakly to a distribution H. Also, H n satisfies conditions 

5) F c ' H (x) has a compact support [a, 6] with a > 0; 

6) the function K(x) and h satisfy 
(1.14) 



nh 2 



x — a 
h 



x — b 
~h~ 



yK(y)dy+ j yK{y)dy -> 0, / y K(y)f" n Hn (yQ)dy < oo 

and 



;i.l5) nh 1 — K(y)dy 



X — a 
h 



x — b 
h 



->0, 



where f Cn ,H„(x) is the density function of F Cn ' Hn (x) and yo = t(x — 
yh) + (1 — t)x with t G (0, 1) and x G (a, b) . 

Then, as n — >■ oo, the limiting finite dimensional distributions of the pro- 
cesses of 

(1.16) nh(f n (x)-f Cn , Hn (x)\, xe{a,b) 

are multivariate normal with mean zero and covariance matrix a 2 I, where 

a 2 = —— / / X'(mi)X'(u 2 ) ln(tti - u 2 ) 2 du 1 du 2 . 

J -oo J -oo 

Remark 1. When T is the identity matrix, $1.10\) is true, which will be verified 
in Appendix 2, and conditions U.ll\) and U.12\) also hold, see, (6.30) in [5] 
and \1.2J$ . For generalT, may be removed at the cost of higher moment 

of H n (t) and of a more stringent bandwidth. See LemmalE in Appendix 2. 

Remark 2. It is easy to check that the Gaussian kernel function satisfies 
conditions specified in Theorem^ 
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Theorem [I] is actually a corollary of the following theorem. 

Theorem 2. When the conditions fli.i^J ), M.15\) and lim^oo nh 3 = in The- 
oremU\are removed with the remaining conditions unchanged, Theorem^ holds 
as well if the processes U.16]) are replaced by the processes 

nh[f n (x) -±j\{^-)dF^ H "{y) 

We evaluate the quality of the estimate f n (x) by the mean integrated square 
error 

rb 



E( / (/„(*) - f Cn , Hn (x)) 2 dx) 



Bias(f n (x))) dx + Var(f n (x))dx 



where Bias(f n (x)) = Ef n (x) — f Cn h u (x). It is easy to verify that (see [H] and 

TO 



1 / TS( X y\JVC n ,H n („,\ P /„ \ _ 1 1,2/ rr.ll, 



h J K(—^-)dF Cn ' Hn (y) - f Cri ,H n {x) = ~h {f c ' {x))" J x K(x)dx + 0(h A ). 

Although it is not rigorous from Theorem [2] we roughly have 

Ef n (x) -\f K(^-)dF^(y) = o{\) 
h J h nh 



and 



a 2 , a 2 



Var(f n (x)) = —- + o 



n 2 h 2 n 2 h 2 
These gives 

l u2 tt /sx// f^r,,^ , ^/,3n , „/ 1 A 2 , cr 2 (b-a) , , a 2 



hVe^W / x 2 K(x)dx + 0(h') + o(—)) + a \, 2 Qj +o(4-). 

Differentiating the above with respect to h and setting it equal to zero, we see 
that the asymptotic optimal bandwidth is 



where c\ = ^(f Cn ,H n ( x ))" J x 2 K{x)dx < oo. This is different from the asymp- 
totic optimal bandwidth 0(l/n 1//5 ) in classical density estimates (see |14j). 

As for F n (x) = Jq f n (y) dy, we have the following result. 

Theorem 3. In addition to assumptions 2), 3), 4) an d 5) in Theorem^ 
suppose that 

Hmn/i Wln- — V oo, lim h — > 0. 

n— s-oo y h n— >oo 
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Then, as n — )■ oo ; the limiting finite dimensional distributions of the processes 
of 



:i.is) -JZ=[F n (x)- 



1 I ff(^)dF*'**(y)]dt) 



are multivariate normal with mean zero and covariance matrix ttsI. 

Remark 3. VKe conjecture that TheoremlMis still true if we substitute F Cn,Hn (x) 

is consis- 



for f*^ [i / K(t-JL)dF Cn ' Hn (y)]dt. The convergence rate n/wln- 

ieni it)it/i i/ie conjectured convergence rate n / \f\og n of the empirical spectral 
distributions of sample covariance matrices to the MP type distribution. 

The paper is organized as follows. Theorem [2] is proved in Section 2 and 
Section 3. In Section 4 we present the proof of Theorem [31 Some technical 
lemmas are given in Appendix 1. Appendix 2 deals with Remark [1] and The- 
orem [1] and Appendix 3 gives the derivation of the variances and means in 
Theorem [2] and Theorem [3j 



2. Finite dimensional convergence of the processes 

Throughout the paper, to save notation, M may stand for different constants 
on different occasions. This and the subsequent sections deal with Theorem 

[2] and the argument for handling nh(j^ K(^ M -)dF Cn ' Hn (y) — f Cn ,H. 

given at the end of Appendix 2. 



x is 



Following the truncation steps in |2] we may truncate and re-normalize the 
random variables as follows 

(2.1) \X i3 \ < rnn 1 ' 2 , EX ij=0 , EX% = 1, 
where r n n 1//3 — > oo. Based on this one may then verify that 

(2.2) EX* 1 = 3 + 0(-). 

n 

For any finite constants h, - ■ ■ ,l r and x±, ■ ■ ■ ,x r G [a, 6], by Cau dry's for- 
mula 

nh(jr i i(fn(xj) ~\J K{^L)dF^{y) 
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where X n (z) = tr(A n — zl)^ 1 — nm F c n ,H n (z) and C\ is defined in the introduc- 
tion. 

From Fubini's theorem and (jl.8p we obtain for j = 0, 1, 2. 



a r 



i r\„u^x-u 



h Jo h 
This implies for u G [«/ , a 



"i 



dv < oo. 



(2.4) - / \K ij) (—— + iv)\dv < oo, j = 0,1,2. 

h Jo h 

For the sake of simplicity, write A = A n . We now introduce some notation. 
Define A(^) = A— zl, Ak(z) = A(z) — s^s^, and = T 1 / 2 x^ with being the 
fcth column of X n . Let = E(-\si, ■ ■ ■ , s^) and E denote the expectation. 
Set 

= 1 i TA-if ^ ' Vkiz) = s^Afc 1 (z)s fe - -trTAj^z), 
i i 

trr - 1 



1 + EtrTA^ (z)/n K 1 + irTA^(z)/n 

We frequently use the following equalities: 

(2.5) A-\z) - A k \z) = -(3 k (z)A k \z)s k s T k A k \z); 

(2.6) ft = 6i - = 6i - 6?£i(z) + &iAei 2 (^) 

where £i(z) = A^f (£)si — En^trAj (z)T. At this moment, we would 
point out that the length of the vertical lines of the contour of integral in ( 12. 3ft 
converges to zero. As a consequence, except we can not expect 

and |/3fc (-2)| to be bounded above by constants although they are bounded 
by |2|/H (see pQ) (of course v ^ in the cases of interest). Instead, the 
moments of (3 k (z) and 13% {z) are proved to be bounded. We summarize such 
estimates in Lemma [3] in Appendix 1. Sometimes we deal with the term 
i-trTA^ 1 (^)TA^ 1 (z) in the following way: One may verify that 

Im(l + UrTA k \z)) > v\ min (T)^trA k 1 (z)A k 1 (z), 
which implies that 

(2.7) \^{z)UrTA- k \z)TA k \z)\ < 
We also frequently use the fact that ||A A T 1 (z)|| < l/\v\. 
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Write 



n 

trA-\z) - EtrA-\z) = (E k trA- 1 {z) - E k _ 1 trA- l {z)^ 

k=l 



tr 



A-Hz^-AtHz) 



k=l 



n 

(^fc-^fc-i) [f3 k (z)slA~ 2 (z)s k 



k=l 



(2.8) 



(Ek-Ek-i) \logp k (z 



k=l 



where in the third step one uses (12.51) and the derivative in the last equality is 
with respect to z. We then obtain from integration by parts that 



(2.9) 



— / K{^—^)(trA-\z) - EtrA-\z))dz 
2ni J h 



2ni 



k=l 



J2(Ek - E k -i) & K{^-^) log&(* 



h 



dz 



(2.10) 



k=l 



dz 



1 1 



k=l 



h ''~°\f3 k {z) 



(2.11) 
where 



1 1 

h 2ni 



n „ 
fc=l ^ ' 



l3k( z )Vk(z) +e k (z))dz 



e k (z) = log(l + - fi(*Mz) 



Below, consider z 6 72, the top horizontal line of the contour, unless it is 
further specified. We remind readers that v = vqH on 72. The next aim is to 
prove that 



(2.12) 



k=l 



-J^iEk-Ek^) I K\^)e k (z)dz^0. 



x — z s 
h 



By Lemma HI we have for m = 2, 4, 6 



(2-13) E(\ Vk (z)r\A k \z)) < -^[hrTA k \z)TA k \z)[ 

This, together with Lemma [3] in Appendix 1 and (12.71) . gives 
(2.14) E\ffi(z) V k(z)\ 4 = E(\ffi(z)\*E(\ V k(z)\ 4 \A k \z))) < 



to/2 



M 

n 2 v 2 
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It follows that 



> 1/2) < 2 4 5>|/3f W?^)| 4 < 



M 



k=l 



k=l 



nv 



2 ' 



Via ([L8D, QUED and tne inequality 



(2.15) 
we obtain 



log(l + x) - x\ < M\x\ 2 , for \x\ < 1/2, 



i I n r 
E\J2(E k - E k ^) K'( 
k=i J 



x — z. 



h 



e k (z)I(\ffi(z)7i k (z)\ < \/2)du 



(2.16) 



k=l 



x — z. 



h 



e k (z)I(\^(z) Vk (z)\ <l/2)du 



M 

7? 



k=l 




\K'(^-^)K'(^-^)\(E\((3i r (z 1 ) Vk (z 1 ^^ 



h ' y h 
xE\^ t k r (z 2 )r lk (z 2 ))\ 4 Y /2 du 1 du 



< 



M 



Thus, (I2.12p is proven. Therefore on 72 

1 n 

(2.17) m = J^y^x) + o p (l) 

where 



k=l 



Y,(x) = E k 



llK 



if X Z \ I atr 



h 



Pl r (z) Vk (z))dz 



Apparently, Y k (z) is a martingale difference so that we may resort to the 
CLT for martingale (see Theorem 35.12 in [1]). As in (I2.16p . by (11. 8p and 
( EED we have 

n M 

£sin(*)i 4 < 



k=l 



mr 



which ensures the Lyapunov condition for the CLT is satisfied. 

Thus, it is sufficient to investigate the limit of the following covariance func- 
tion 



4vr 2 



1 n 

-j^£ fc _i[n(zi)y fc (a; 2 )] 



k=l 



(2.18) 



Ah 2 n 2 
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where 

n 

C n M,z 2 ) =^£; fc _ 1 [ J B fe (^(^ 1 ) % (^ 1 )) J B fe (/3f(z 2 )77 fe (z 2 ) 

k=l 

By (JZZD, (J2I3D and O 

E{\(^(z)-b 1 (z))r ]k (z)\ 2 

< —El \(5 t ;{z)b l {z){trA- 1 (z)T - EtrArHzYT) 

(2.19) < -^l^f^ll^A^^T-MrA-^^Tl 2 . 

This and Lemma [3] lead to 



x-trTA- k \z)TA k \z) 



n 



M 



E 



(fifa) - b 1 (z 1 )) Vk (z 1 )E k [(^(z 2 ) - b l (z 2 ))r ]k (z 2 ) 



< 



and 



E\(f3f( Zl ) - b 1 (z 1 )) Vk (z l )\ 2 E\(^(z 2 ) - b l (z 2 )) Vk (z 2 )\ 2 

M 



1/2 M 
< 



E 



< 



2 7 .5/2 ' 



It follows that 



(2.20) E C n (z l ,z 2 )-b 1 (z 1 )b 1 (z 2 )^E k ^ 1 (E k r ]k (z l )E k T ]k (z 2 ) 



k=l 

Note that for any non-random matrices B and C 
£(sfCs! - trC)(sfBsx - trB) 

-2, 



< 



M 



nv 



5/2 ' 



n 



l (EX^ - \EX 2 U \ 2 - 2) ^(T 1 / 2 CT 1 / 2 ) rj (T 1 / 2 BT 1 / 2 ), 



i=l 



(2.21) +\EXl 1 \ 2 n-HrT l l 2 CTB T T 1 ' 2 + n^rT^CTBT 1 / 2 . 
This implies that 

n 

y^^fc-i [ E k T] k (z 1 )E k T] k (z 2 ) 
k=l 

(2.22) = (EX ft - 3)6i(^)6i(«2)C;i(zi, as) + 26 1 (^ 1 )6 1 (z 2 )C n2 (z 1 , z 2 ) 



(2.23) 



26i(«i)6 1 ( < 82)C n2 («i, + 0(— ), 
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where 



n p 

C nl {z u z 2 ) = -^^(^(T^A-^zOTV^^.^^A- 1 ^)^/ 2 ),,, 



n 

k=l j=l 



1 - 

C n2 (z u z 2 ) = -J2trT 1 / 2 E k (A k \z 1 ))TE k (A k \z 2 ))T 1 / 2 , 

k=l 

and in the last step one uses Q22J and the fact that \(E k (T 1 / 2 A k 1 (z 1 )T 1 / 2 ) jj \ < 



The next aim is to convert the random variables involved in C n2 (zi, z 2 ) to 
their corresponding expectations. To this end, we introduce more notation 
and estimates, and establish a lemma. Define 

A kj (z) = A{z) - s fc s£ - s jS j, f3 kj {z) = 1 , 

1 + Sj J\ kj [Z)Sj 

1 1 

6l2 W= l + n-^EtrTA- 2 \zy = 1 + n-HrA^(z) 

and 

£ kj (z) = sjA k }(z)s^En-HrA k }(z)T, Vkj (z) = sj A k l(z)s ] -n- 1 trA k }(z)T. 
Note that 

(2.24) A- k \z) - A k j{z) = -(3 kj (z)A k }(z)s ]S jA k }(z) 
and (see Lemma 2.10 of pQ) for any p x p matrix D 

(2.25) | tr (A- 1 (^)-A fcj 1 (z))D|<M. 

v 

By Lemma [3] in Appendix 1 and (I2.25P we have 

(2.26) ^E\trA- k j{z)T - EtrA^{z)T\* < ^ ff , E\^(z)\* < 
By Lemma [3] we have 

(2.27) E—trA~ 1 (z)A~ 1 (z) = -ImiE-trA^U)) < — , 

n v n v 

which, together with (I2.25p . implies that 

(2.28) E^-trA k j(z)A k j(z) < ^. 

By (12.251) and the fact that bi {z) is bounded, given in Lemma [3J it is straight- 
forward to verify that \b\{z) — b 12 (z)\ < and hence 

(2.29) \h 2 (z)\<M. 
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We are now in a position to state Lemma [U 
Lemma 1. For non-random matrix D 
E 



hrBA-\z 1 )TE k (A-\z 2 )) - EQ-trBA k \z 1 )TE k (A k \z 2 )) 



(2.30) < 



#5 when IIDII < M 
■M* when itrDD* < M. 



Remark 4. Checking on the argument of Lemma [21 we see that Lemma [J\ 
holds as well when we replace E k (A k 1 (z 2 )) by A^ 1 (2 2 ). The main difference 
of arguments is that we do not need to distinguish between the cases j < k and 
j > k when dealing with the latter. 

Proof. We begin with a martingale decomposition of random variable of inter- 
est: 

■^-tr~DA k ~ 1 (zi)TE k (A k ~ 1 (z 2 )) - E{hr"DAl\ Zl )TE k {Al\z 2 )) 
i n 

n Yp s - [trBA k \ Zl )TE k (A k \z 2 )) 

- Ei-i) [irDA^CzOTSfcCAj 1 ^)) - trBA^( Zl )TE k (A^(z 2 )) 
1 n 

= -J2(E j -E j _ 1 )(5 1 + 5 2 + 5 3 ), 
n &k 

where, via (12.241) . 

Si = /3 fc ^^)sjA fc 7(^ 1 )m(/3 fc ,^ 2 )A^ 1 (z 2 )s J sjA fc 7(z 2 ))DA fc - 1 (^ 1 )s J 

6 2 = -/3 fcj (^i)sjA fc - 1 (^ 1 )TE fc (A fc - 1 ( 22 ))DA fc - 1 (z 1 )s, 

and 

5 3 = -trDA fc 7(^ 1 )m(/3 fcj (z 2 )A fej 1 (z 2 )s,sjA fc - 1 (z 2 )' 

Note that 

(2.31) I^HIsjA- 1 ^)!! 2 = \P kj sjA;l(z)A;l(z) Sj \ < ± 
which implies that 

(2.32) 1^1 < ' 



v 2 WD\ 
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Write 
(2.33) 

0kj(z) = h 2 (z) - p kj (z)b 12 (z)£ kj (z) = b l2 {z) -b 2 12 {z)^ kj {z) + (5 kj (z)b\ 2 (z)^ 2 kj {z). 
This implies that when j > k, 

{E j - Ej^S, = {Ej - E^)b 12 ( Zl ) (6 U - 5 12 ), 
where 5i 2 = £fcj(zi)5i and 

-n-HrTA k }(z 1 )TE k ^ kj (z 2 )G k (z 2 ^BA k }(z 1 ) 

with G k (z 2 ) = A k j(z 2 )s j sjA k '}(z 2 ). When ||D|| < M we conclude from ([273TD . 
(I2~32l) . (E2BD and Lemma S that 



1 n 1 n 



M 



/ A y 7 J- / \ ±-L i ~±^7| — : o / , " rii 1 " J--^ — : 9 ^IIt-xIIO 

n z — 4 n z z — ' nV U r 
When itrDD* < M, by Lemma Sand PUT]) 

E 5 n 8 < — : — — (— trDD ) < 



n A v 2i n n^v 24 
This, together with f !2.26|) and Lemma [3J implies 

E\5 12 \ 2 < ME\i k3 f3 k M 2 +^ E \^ 3 h ] trTA k }{z l )TE k ^ 

M 
< . 



nv° 

because via (I2.3ip and Holder's inequality 

^|trTA- 1 (^ 1 )T^(^(^)G fe (^))DA fc - 1 (z 1 )| 8 

(2-34)< ^E\ l -trA k j{ Zl ) - eUtA^z^ + ^\E l -tr A^ { Zl )t < ^. 



These give 

1 1 . " M 

£| - Y>y - + 5 12 )| 2 < - V £|5 n | 2 + ^|5i 2 | 2 < — 
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For handling the case j < k, we define A k j(z),P k (z) and C k j{ z ) using 

si,--- ,Sj-_i,s i+1 , ••• ,s k -i,§.k+i,--- ,§n as A~^(z),p kj (z) and £ kj {z) are de- 
fined using si, • • • , Sj-x, s j+1 , ■■■ , s fc _i, sjfc + i, • • • , s n . Here s 1( • • • ,s n are i.i.d. 
copies of Si and independent of {Sj,j — 1, • • • , n}. Let 

afci = sjA-/(z 1 )TA fe /(z 2 )s„ a k2 = sjA fc /(z 1 )DA-/(z 2 )s J , 

Applying (I2.33f) and the equality for /3 (z 2 ) similar to (12. 33ft yields 

— KJ 

{Ej - Ej-x)5 x = (Ej - Ej-i) p kj (z 1 )p k .(z 2 )a k ia k 2 

= b 1 2{z 1 )bi2{z 2 ){5i i + 8 1A + 8 X5 + 8 X 6 + 5x7 + 8xs), 

where 

Sxs = (Ej — Ej-x) ((kji(kj2) , ^14 = (^-^--O^-iti-^A^^ODA^^t), 
8 15 = (E 3 - ^_ 1 )(a i2 n-VA^ 1 (^ 1 )TA^ 1 (^)T 
8\& = ~{Ej - Ej_x) Pkj{zi)^kj{zi)aki<y k 2 
8x7 = ~{Ej - §_ kj {z2)§_ kj {z 2 )a kl a k 2 

{Ej-Ej_x) f3kj{zi)^ kj (z2Kk J {zi)^ k .(z 2 )a kl a k 2 



and 



with 



'18 



-kj 



Cm = a fcl -n- 1 trA fe /(^ 1 )TA-/(^)T, ( kj2 = a k2 -n- 1 trA k ^{z 1 )BA k ^z 2 )T. 

Consider ||D|| < M first. It follows from Lemma U fl2T26l) and (E2HD that 
(or see f !5.8j) in Appendix 1) 
(2.35) 

£|G;i| 4 < -^ i ^|-*rA- 1 (^ 1 )A- 1 (z 1 )| 2 < ^ia, 2 | 4 < 2 ff n||a . 

n^u 4 n J J nv n z v°\\D\\ z 

Similarly, by fl2T26|) and f l2T28|) . as in fl2T34j) . we have 
E|n- 1 trA fe - 1 ( 2l )DA- 1 (^ 2 )T| 4 

w (E|n- 1 trA fc 7(,0A fc 7(^ 1 )| 4 ^|n-HrA^(^ 2 )A fe - 1 (^ 2 )| 4 ) 
M 



< 



(2 - 36) " ||D||V 
In view of f l2T35|) and fl2T29|) . 



1 n 

Sl-J]^ -^•_ 1 )6 12 (z 1 )6 12 (z 2 )(5 13 )| 2 < 



M 

n 3 v 6 \\~D\\ 2 
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While (ESED and ( l2~29l) yield 

E ^J2^ -^-OM^M^)! 2 < 2 ™ , J =4,5. 
It follows from f EPTD that 

IMzijaktatal < — \\A^ J 1 {z 2 )s j \\ 2 = —sjA^ J 1 (z 2 )A^ J 1 {z 2 )s j . 
In the mean time we obtain from Lemma H] 

(2.37) E|sjA- 1 (,- 2 )A- 1 (, 2 )s J - itrA fc 7(,- 2 )A fc 7(, 2 )T| 8 < JL. 

Thus we have 

M 



■ n i — t - - n 2 v 5 

Obviously, this estimate applies to the term involving 5 17 . From (I2.3ip and 
Lemma [3] we obtain 

E\-j^{E j -^•- 1 )& 12 (^)6 12 (z 2 )(5 18 )| 2 < 3 6 j l >||2 . 

Summarizing the above we have 

1 " A/f 

Consider ^trDD* < M next. By Lemma H] 
(2.38) 



< ^^(^rA fc 7(, 1 )DA-(, 2 )TA-(,- 2 )DA fc 7( % )T) 4 < -^(1 
Observe that 

(2.39) E\n- 1 trA k }(z l )BA- k l(z 2 )T\ 8 < -^(n^rDD*) 4 . 

This, together with fl235D and (Ell, give 



and 



1 11 M 
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To deal with <5i 6 , we obtain from (I2.3ip 

E\P k j{zi)ikj{zi)a k ia k2 \ 2 < ^E^\f3 kj (z 1 )\^ kj (z 1 )C kj2 \\ A ~ 1 t> 2 ) s/ A 



+^E(^|/3 fc ^2 ; 0|^<^l)l|A^ 1 (2 ;2 )s,||n-4rA fc 7( ^ 0DA^(^)T)^ 



M 
v 

We conclude from f[2~38j) . f l2~26|) . 0371) and Lemma [3] 

2 



# ( \/\Pkj(Zl)\£kj (Zl ) C*i2 1 1 A fc / ( Z 2 ) | 

< (^l^i^iC^I 8 ) 174 ^!^-! 4 ^IsjA^^A^^s,! 4 ) 1 / 4 < u 



n 2y6 



Replacing ( k j 2 with n l trA k ^{zi)Y)A Lk j{z 2 )T in the above similarly gives 

2 M 



^(Vl^(^i)l6 J (^i)l|A fc - 1 (z 2 )s J ||n- 1 trA fej 1 (z 1 )DA fc - 1 (^)T) < 



nw 5 



Consequently £|~ ^ (J5,- --£j_i)&i 2 (>i)&i2(>2)(£i6)| 2 < Jftr- This estimate ap- 
parently applies to the term involving 5i?. Also, for 5i$ we similarly have 

E W ^ ^^(7l^2)|^^i)^(^i)l|A fc - 1 (z 2 )s J ||) 2 < JjL. 

The terms 5 2 and £3 can be similarly, even simpler, proved to have the same 
order. Thus Lemma [D is complete. 

□ 

Combining (|Tg)) . f l2TT8|) . fl2T20|) . f l2T23|) and Lemma[I]with D = / we conclude 
that 



1 n 

— J^Ek-ilYkix^Y^)] 



Air 2 

k=i 

rl Xi — Zl K l/ X 2 — Z 2 , 



(2.40)= 0T2 2 / / ^( J -r^)^(- L r^) a »i (zuzddzidzt + qp(l), 



where 



&i(zi)&i(z 2 ) 

fc=i 

Thus, it is enough to investigate the uniform convergence of a n i(zi, z 2 ). 
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2.1. The limit of a n \(zi, z 2 ). Before developing the limit of a n i(zi, z 2 ), we 
first establish Lemma [2] below, which improves (15 .ip in Lemma 

Lemma 2. 

(2.41) ^-E\trA^ l (z)F- 1 (z)'D - EtrA^^F- 1 ^)^] 2 < U 



n 2 t> 3 ||D|| 2 ' 



where F'\z) = {Em n T + I)" 1 and D = T or (zl - r -^b 12 (z)T 



-1 



Proof. With notation F 1 1 (z) = [zl — IL ^-b 12 [z)T) l , we start the proof of 
Lemma [2] by presenting the equality (2.9) in [2] 

(2.42) 

where 



A k \z) = -F^[z) + b X2 [z)B[z) + C[z) + D(z), 
B[z) = ^Fr 1 W(s,sJ -n- 1 T)A^(z), 



and 



D(,)=n-% 2 (,)Fr 1 WT^(A fc - 1 (^)-A fc 1 (,)). 
By (11.111) we have 



(2.43) 



-trF7 2 [z)F7 2 [z) < M, -trF~ 2 [z)F~ 2 [z) < M. 

n n 



However sometimes we also use the fact that (see (2.10) in [2] and Lemma 2.11 
of P) 



(2.44) 



M „„, ,„ M 
Fi (z <— , F(z <— • 



With H = F ^2) or H = I, we first apply (12.421) with z replaced by z 2 to 
prove that 
(2.45) 



1 



-E 



n 



trA^ 1 (2 2 )HTFr 1 (^i)T 



-trF^(z 2 )UTF^( Zl )T + 0(— ^) , 



which, together with (I2.43p . then implies that 
(2.46) ' 



—E 

n 



trA- k \z)WFF- l \z)T 



< M. 



To see $233]), first note that 
1 



-E 



n 



trB[z 2 )WTF^[z x )T 
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Second, applying f !2.33f) yields 
1 



-E 



n 



trC(z 2 )UTF^ 1 (z 1 )T\ = --^^ni^Vkji^Vkji 



■-^E\bUzi)-(trA^(z)T-EtrA^(z)T)-^ 
n t —~ l n J J n 



where 



+ -^Ek^^)^ 2 (z)^sjA fc - 1 ( 2 )HTFr 1 (^ 1 )TFr 1 (z)s J 

U o+k 



Vkjl = sjA~ k Hz)IlTF^(z)TF^(z)s J - — tr A7 1 (z) HTF7 1 (z) TF7 1 (z)T. 



n 



By Lemma H flCTj) and (T2744j) 



(2.47) 



1 ^ ' trFi 2 (z)Fi 2 (z)) 2 < 



n 2 v 8 n 



n 2 v s 



and via Holder's inequality and ( I2.43P 
(2.48) ' 



n 



< 



M 



Appealing to (l271Tj) and ( 12726]) yields 



< 



M 



nv 



5/2 ' 



We obtain from (12718]) and (127261) 



J J n 

In view of (1274T]) . ( 12726]) . (12718]) and Lemma [3] we have 



< 



M 



nv 



5/2 ' 



E 



P kj (z 1 )e k ^A-l{z)WFF- l \z 1 )TF- l \z l )s 



M 



nv 



5/2 ' 



Let equal to 

sjA fc 7(^)HTFr 1 (^)TFr 1 (^)A fc 7(^)s,-^rA fe 7(^)HTF7 1 (^)TFr 1 (^)A fc 7(^)T. 
Then, as in (I2T4"7) . 

(2.49) E|^ 2 | 4 < 



M 
n 2 v 12 



n 



trA^(z)HTFi 1 (z)TFi 1 (z)A^(z)T 



k,i 



< 



M 



This, together with ( 12.33!) . 1 )2. 26 ft and Lemma El ensures that 



-E 



n 



trD(z 2 )UTF^ L ( Zl )T 



< 



M 

12,3/2^5/2 
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Thus (E35D is true. 

When replacing F[" (2), T with F _1 (z), I, respectively, (I2.46P further ensures 
that 
(2.50) 

EUr¥-\z)A k \z)A k \z)¥-\z) = ^^trF-\z)A k 1 (z)F^(z)) < 



M 

v 



As before, we get a martingale representation as follows: 
hrA k 1 {z)F- 1 {z)B - EtrA- 1 {z)F- 1 {z)B 

l - J2(E j - E^) (trA k \z)F-\z)F> - tr A k } {z)F-\z)F>) 
\ J2(Ej ~ E^ 1 )sjA-l(z)F- 1 (z)rjA k l(z)s J p kj 



n 



n 

3^k 



71 i*k 

where 

1 



h = s 1 J A k ;(z)F-'(z)BA k ;(z)s J - -trA k ;(z)F-\z)VA k }(z)T, 

02 = -b 12 (z)sjA^(z)F- 1 (z)rjA k }(z)s J ^(z), 

and 

03 = b 12 {z)h,^A k :l{z)F-\z)F)All{z)s j e kj {z)- 

Here in the last step one uses (12.33!) . By Lemma H] and (12.501) we have 

£|0i| 2 < ^^A^^F-^^DA^^TA^^DF^^AjJ^) 

(2-51) < Jp^^A-^A-^F- 1 ^) < 

Similarly, from (12.431) and (I2.28f) it is easy to verify that 
(2.52) E\<f> x I 8 < l/ 



nV 4 ||D|| 8 

As in ( 12~13|) . by Lemma El ( 12T261) and fl230|) we have 



E\-trA k }{z)F-\z)DA-}{z)Tr lk3 {z 
n J J 



^^^(l^rA^^F-^^DA^^TpitrA.-^^A,- 1 ^ 
itrA^(^)F- 1 (,)F- 1 (^)A fc - 1 (^)(itrA fc - 1 (^)A fe - 1 (^ 



M ^ 
< — — —E 

~ n D 2 
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M 



< 



n 



3 t» 2 ||D|| 2 



E 



trA-}(z)F^(z)F- 1 (z)A-l(z 



i- j 



trA-Uz)-EtrA-Uz) + \EtrA k f(z^ 



kj 



kj 



< 



M 



nf 3 ||D|| 2 



(2.53) ) 
Note that 

{E S - E^) \hrA^(z)F-\z)VA k }(z)T(hrA-}(z)T - ±EX£(z)T) 
.n n J n J 

This, together with (12321) . fl2~53|) and (j2261) . implies that 

E\-J2(Ej - E^M 2 < - ^ 
n z — ' nV D r 

By (12.521) and Lemma [3] we obtain 

(2.54) m^)\ 8 )nE\h\ s mf)^ < J^im^ < M 



rrv 



3 7 ,13/2 ' 



n ,J v 



From Holder's inequality, (E2ED and (B3SD 



n J J n J J n 

This, together with Lemma [3] and (12.26j) . implies that 

{E\UtrA-Uz)-EtrA k Uz))\*yl\E\UrA k ^ 



< 



M 
n 4 v 10 



and that 

{E\htfYl\E UrAl}{z)¥-\z)VA^{z)T\% kj {z)\* )W 

^ ^(^I^A^WF^WDA^^TitrA^Cz) V /2 < M 
These and (12.54j) ensure that 



n 2 v 5 



E\h\ 2 < 



M 

n 2 v 5 



Thus, Lemma [2] is complete. 



□ 



To later use, we now consider a more general form than a n i(z\, z 2 ) 

1 n 

(2.55) a^( Zl ,z 2 ) = -J2E[trTA k 1 (z 1 )TE k (A k \z 2 ))U_ . 



k=l 



One should note that a^ 2 ( z i> z 2) reduces to a n2 (zi, z 2 ) when H = I. 
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Applying the definition of C(z\) and (12. 33ft gives 
1 



(2.56) 
where 

and 

Here 

(2.57) 

Define 



-E 



n 



trTC(z 1 )TE k (A- 1 (z 2 ))H = d(^) + C 2 ( Zl ), 



CM) = -b 2 l2 (z l )-J2 E [^kj(zi)sjA k ^(z 1 ,z 2 )s 



3+k 



A^ k (z u z 2 ) = A k }(z 1 )TE k (A k \z 2 ))llTF^(z l ). 



kj 



( kj3 = Sj A kjk ( Zl , z 2 ) Sj - -trA k j k ( Zl , z 2 )T. 

Consider j > k first. Then by Lemma H] and (12.431) 

M 



(2.58) 



^ICfej3| < 



and via an argument similar to fl 2 . 3 6 1) . (I2.28P and Holder's inequality 
(2.59) 



^ A ^W 2 )T| 4 <^. 



It follows from d2T58|) . f[239j) and fl2T26|) that 

l^i)|<^E^^i^)| 4 ) 1/2 [^l^( 



3+k 



X 



E\( kj3 \* + E\-trA k f k ( Zl ,z 2 )T 



n 



1/4 M 

< 



n v 



5/2 ' 



As for d(zi), by (12381) . ( I2~26l) . (12T25D and ( 12301) with D = HTF^fcOT 
have 



j>fe 

+-^(irAD L - £irA^)(*rAr^ 1; z 2 )T - EtrA k Uz u z 2 )T)\ = 0{ ' 



rr 



L fci -^ Vl -™*kj )\ VI ^kjk\ 

When j < fc, decompose A^ 1 (2 2 ) as 

A k j{z 2 ) - A k j{z 2 )s^A k \z 2 )p kr 



nv 



5/2- 
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Then, apparently, the above argument for the case j > k also works if we 
replace E k (A k 1 (z 2 )) in A~j k (z l} z 2 ) with E k (A~^(z 2 )). For another term of 
C 2 (*i) by (gSTED , (TJ^nD and Lemma 



/3^0^ 2 )^.(z)sjA^^^^ 



< ^(£|^^0| 2 ^l^/^)r£|^| 8 J E|sjA^(z 2 )HTFr 1 (z 1 )s,| 4 )^ < J* 



Ti V 111 



because 
(2.60) 

with = sjA fc /(2 2 )HTF^ 1 (2 1 )s j - n- 1 trA fej 1 (2 2 )HTF^ 1 (^ 1 )T. As for an- 
other term of C\{zi), an application of (12.331) yields 



j<k 



J2 E k fcj ^ 2 )^(^i)sjA fc - 1 (z 1 )TA fc - 1 (z 2 )s J sjA fej 1 (^)HTFr 1 (2 ;i )s J 



(2.61) = -ii V[C n + Cia + Cia + C 14 + C 15 + C 16 ] 



i<fc 



where 
C U = E 



£,kj{Zl)(kjl(kj4 



Ciq = -E 



^(^i)a,i^- 1 trA^ 1 (^)HTFr 1 (zi)T 



Cl3 
Cl4 



Cl5 

and 

Cl6 



^(^i)C^4n- 1 trA-/(2; 1 )TA fc /( 2;2 )T 
^(^)n- 1 trA fc - 1 ( 2l )TA fej 1 (^ 2 )Tn- 1 trA^(^)HTFr 1 (^ 1 )T 



n 



(frAr, 1 ^) 



-EtrA fc - 1 (^ 1 ))n- 1 trA-/(z 1 )TA fc /( 2;2 )Tn- 1 trA fe -/( 2;2 )HTFr 1 (^)T 



-E 



^ i (^)i fci (^)^(^i)sjA- 1 (zi)TA^ 1 (0 2 )s i a i 4 



^^•(^)^ 1 ^2)6i(^)sjA- 1 (z 1 )TA^(z 2 )s i n- 1 trA7 1 (z 2 )HTFr 1 (z 1 )T 



Appealing to f[2T36j) . f T2T35|) . fl2T26|) and f l23U|) yields 

M 



|Cy| < 



nv 



5/2 : 



J = 1,2,3. 
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By (E26]), ( Eg} , fl^26|) and ( ICTj) we obtain |C 14 | < We conclude from 

flf3T|) . flOBj) . flOTj) . Lemma[3]and fl^60|) that 



< C^' c \/'^^7^^ 1 M s J^^^/c^i) ii) 4 ^ie^-c^) i^^ie^-c^i) r^ic^-^r 



1/4 



< 



M 



72,3/2^5/2 



Similarly 



|C 16 | < 



M 



nt ,5/2 

Summarizing the above we have proved that 

(2.62) -E 

n 



trTL7(^ 1 )T J E; fc (A fc 1 (^))H 



< 



M 



nv 



5/2 ' 



Consider D(z\) now. When j > k using (I2.33|) and recalling the definition 
of A^j k (z 1 ,z 2 ) in (I2.57P we obtain 



-E 



n 



trTD(z 1 )TE k (A k 1 (z 2 ))H 



: b 12 (z 1 )J2[Di + D 2 +D 3 



where 



D l = E 

n 



trA^ k (z 1 ,z 2 )TA^(z 1 )T 



D, = E 



and 



D 3 = -E 

n 



trA^ k (z ll z 2 )TA^(z 1 )T^ kj (z 1 )/3 kj (z 1 ) 



with 



- sjA^^TA^s,- - -trAj^^TA^*, )T. 



Using fl2T43|) . (12^51) and Holder's inequality 
1 



n 



(2.63) 

By Lemma 2.7 [1 J and fl2T43|) 

(2.64) 
Thus 



trA-} k (z h z 2 )TA-l( Zl )T 



kj 



2 M 
< . 

t> 5 



, , M , „ . M , „ . M 
l^il < -575, < — , \D 3 \ < 



■\/nv 3 
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Hence when j > k 



-E 

n 



trTD( Zl )TE k (A k l (z 2 ))U 



< 



M 



n t: 



5/2 ' 



When j < k, divide A^, 1 ^) into the sum: 

Afc/(z2) - A k j(z 2 )s j s T J A k j{z 2 )f3 kj {z 2 ). 

Apparently, the above argument for the case j > k also works for the term 
involving E k (A k j(z 2 )) if we replace E k (A k 1 (z 2 )) with E k (A k j(z 2 )). Another 
term is 

xsjA-^^^HTFr^^OTA-^^Os,- 

which has, via ((EH]), (ESHj) and 1)555} with D = HTF^ 1 (z :L )T, an order of 
i 

Thus, the contribution from C(zi) and -D(^i) is negligible. 
Next consider B(zi). It follows from (I2.24p that 

(2.65) ^-E^rTB(z 1 )TE k (A k \z 2 ))U 

lj2 E [ s J k kU z ^ z *>j -n-HrTA- k j k (z 1} z 2 )\ = B 1 (z 1 ) + B 2 (z x ), 



n 

j<k 



where 



= -^E^^/^^sjA^l^OTA^l^^s.sjA^^^^HTFr^^Os, 

j<k 



and 



= -^E^^/^^sjA^^^HTFr^^OA^^OTA^^^s, 



j<k 

For B 2 (z\) we further write 

B 2 (z 1 ) = B 21 (z 1 ) + B 22 (z 1 ), 

where 



^i) = -^^E^[^rA^(z 2 )HTF r 1 (^ 1 )A^(z 1 )TA^(z 2 )T 
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and 



B 22 (zi) 



612(22) 



E%>)i fc >) 



j<fc 



xsjA fc - 1 (. ;2 )HTFr 1 ( 2l )A^(^)TA-/( 2;2 )s 



The inequality similar to (I2.63j) ensures that |£? 2 i(zi)| < 



A 7 



^72, while |.B22(2i)| ^ 

n 3 /2 ^ 3 by estimates similar to ( I2.63P and (12.641) . Therefore B 2 {z\) is negligible. 
By (1X331) . (1235|) and the estimates of dj, j = 1, 2, 3, 4 in fl23T|) we have 
612(22) 



5l(2 X ) + 



E [sjA fe - 1 ( 2;i )TA fc - 1 ( 2;2 )s J sjA fe - 1 (^)HTF r 1 (^ 1 )s J 

j<fc 

= <><:"• 



In the mean time, (12.351) and ( 12.601) ensure that 



lX, E [^A fc - 1 (z 1 )TA fe - 1 ( 22 )s,sjA fe 7(^)HTF r 1 (^ 1 )s J 

j<k 

^E[trTA fcj 1 (^ 1 )TA fe - 1 (^)trA fe - 1 (z 2 )HTF r 1 (z 1 )Tj + 0( 



M 



j<k 



nv 



5/2 >' 



Furthermore we apply (12T30D . (EH]), (E25}, (I2T331 and (12311 to obtain 
ii?[trTA fc - 1 ( 2;i )TA fcj 1 (^)trA fc - 1 ( 22 )HTFr 1 (2 1 )T 



rr 



trTA^^ 1 )TA fcj 1 (z 2 )jE[trA fc - 1 (^)HTFr 1 ( 2;i )Tj +0( 
In addition, by (12T25D . ( 12381) . (12~39D . (PX33D and d2~46l) we have 

1 " \trTA^( Zl )TA^(z 2 )] E [trA^HTFr^T 



M , 
n 2 f 5 ' 



rr 



rr 



nv 



It follows that 
(2.66) 



B x { Zl ) + ^6 12 (^ 2 ) J E;(trTA fc 1 (^ 1 )TA fc 1 (^ 2 ) 



x J E^rA fe 1 (^ 2 )HTFr 1 (^i)T 



< 



M 



nv 



5/2 ' 



Summarizing the argument from (12. 56ft to ( 12. 66ft yields 



(2.67) -E 

n 



trTK l (-~i)^E k (A- k \z,))m 



- l -E 

n 



trA fc 1 (z 2 )HTF^ 1 (^ 1 )T 
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J — 1 - 



b u (z 1 )b u (z 2 )E[trTA-\z 1 )TA k \z 2 ))E(trA- 1 (z 2 )UTF^(z 1 )T 



+o(- 



1 



'nv 5 / 2 

When H = I, and ( 1^451) produce 

1 



1 



lJ- 1 -b 12 (z 1 )b 12 (z 2 )-trF^ 1 (z 2 )TF^ 1 (z 1 )T 

n n 



(2.68) =-trF r 1 (, 2 )TF r 1 (, 1 )T + 0(^). 

By the formula ( see (2.2) of p]) 

1 n 

MnO) = — y^pkiz) 

zn L — ' 



fc=i 



we have 

(2.69) = -zEm n (z) 

It follows from (12.61) that 



nv 



and from (jZj25j) that 



|6i(«)-6i 2 («)| < 



These, together with (I3.8p . imply that 



(2.70) 



M*)-sMI< 



M 



M 



nv 



3/2 ' 



This, along with flggg) , (12T28|) and f l2~25|) . yields that 



n 



(2.71) 
where 



Cvkn\ Z li Z 2 ) + 0( ^ N 



1 - - — -b n (zx,z 2 ) 
n 



ZiZ 2 m^(z 1 )mP n (z 2 ) nv 5 / 2 



b n (zi, z 2 ) = c n rrf n (z 1 )rrf n (z 2 ) 
It follows that 



t 2 dH n (t) 



(l + tmO^OXl + tmO^)) 



(2.72) 



a nl 



(*1,* 2 ) = K{Zl, Z 2 )- V- — y— T + 0( ,. 

n f-f 1 - Li6„.(zi, z 2 ) 7 



j=l n 
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From (2.19) in [2] and the inequality above (6.37) in [6] we see that 
(2.73) 

\l-Z^—b n (z u Z2)\>Mv, \l-tb n (z 1 ,z 2 )\>Mv, for any te[0,l}. 
n 

It follows that 

l <A l r l l m 

n j^x 1 - 3j ^b n {z u z2) Jo l-tb n (z 1 ,z 2 ) ~ nv 2 ' 
This ensures that 



anl(zi,Z 2 ) = b n (z!,Z 2 ) I — — r + 0( 



o l-tb n (z 1: z 2 ) nv 5 / 2 
-\ og (l -b n (z l7 z 2 )) + 0(^) 

(2.74) = -]og((*i-^K(*i)f^ ' 



nv 5/2 ' 

where in the last step one uses the fact that by (11.31) 

_ n&(zi)-rn°(z2) 

Z\ - Z 2 — r 7- —{1 - O n [Zi, Z 2 . 

ml{zx)in^ n {z 2 ) 

So far we have considered z e 7 2 . The above argument evidently works for 
the case of z G 71 due to symmetry. To deal with the cases when z belongs 
to two vertical lines of the contour, we need the estimates (1.9a) and (1.9b) of 
j2], which hold under our truncation level. That is 

(2.75) P(||A|| > Mi) = o{n~ l ), P(A^ in < /i 2 ) = 0(71"'), 

for any fii > limsupT(l + \/c) 2 , fx 2 < liminf T(l — y/c) 2 and /. This implies 
that 

(2.76) P(||A fe || > jiO = o(n- 1 ), P(A^ n < » 2 ) = 0(71"'). 

n 

Let B = f] B k where B k = (ai — n < A m f n < ||A fc || < a r — rf) with 77 > so 

k=l 

that a r — rj > limsup T(l + \fc) 2 and ai — n < liminf T(l — \fc) 2 . Also define 
B n+ i — (ai — r] < A^ in < || A || < a r — rj) and let C k = B k n B n+1 . It follows 
that on the two vertical lines 72 U 74 (I2.10p is equal to 

= -r^E^-^-i) f K'(^) log /3 k (z)dz 

71 k=l 

(2.77) = -I-L^^-Efc-O / K'(^)log(3 k (z)I(C k )dz + o p (l). 
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We then introduce j3 k (z), a truncated version of f3 k (z). Select a sequence of 
positive numbers e n satisfying for some G (0, 1), 

(2.78) e n i0, e n >n-P. 

Define 

li = {ai + iv : v G \rT l e n , v Q h] U [-v Q h, -n^En]} 

and 

7 r = {a r + iv : v G [n~ 1 e n , v h] U [—v h, — n _1 e n ]}. 

Write 7n = 7r U 7;. We can now define the process 
(2.79) 

Pk{z) = { ^^PkM + z^PhM, if u = a r , v G \-n- l e ru rT x e r 
m £r L Pk{zii) + £ -^Pk{zi2), if u = a h v G [-n- 1 ^, n" 1 ^], 



where z T \ = a r + in e n , z r2 = a r — in e n , zn = aj + in e n , z\i = a\ — 
in- l s n . Note that ||(A fc - zl)- l I{C k )\\ < ±, ||(A - zl)- l I(C k )\\ < ± and then 
\(3 k {z)I{C k )\ = |1 - s£(A - ^-^/(Cfe)! < Msfs fe . It follows that 

(2.80) P(|Q fc | > 1/2) < 2£wjE;(S ^ Sfc) -> 0, 

n 

where Q fc = P k (z)(l/ P k (z) - l/$ k (z))I(C k ). By fl2TT5|) and (T22D we obtain 
j^X>*- S *-i)/*'(^)( lo g^ < \)dz 



k=l 

< Me2 



n 2 



fe=i 

This, together with (12 .77ft and (12.801) . ensures that 



717 fe=l ^ 



fe=i 

where @ k {z) is similarly defined according to /?fc(z). Moreover, for the trunca- 
tion versions, the higher moments of A^ 1 ^), (z) and A~^.{z) are bounded 
(see (3.1) in [2J). Also, as pointed out in the paragraph below (3.2) in [2], the 
moments of j3i(z), (3n(z), (3 tr (z), sj Aj~ (zi)TAi (z2)si are bounded as well. 
Using these facts, all the estimates holding for z G 71 U 72 also holds for the 
case where z = z r ±,z r 2 or z G j r U 7^. Via these facts, the arguments of the 
cases z = z r i,z r2 or z G 7 r U 72, two vertical lines, can follow from those of 



CLT OF NONPARAMETRIC ESTIMATE OF DENSITY FUNCTIONS 29 

the case z G 71 U 72 (here we omit the details) and hence their limits have the 
same form as (I2.74p . 

In the mean time, appealing to Cauchy's theorem gives 
(2.81) 

1 jf jf K'(^^)K'{^^) log {[z x - z 2 )ml(z l )ml(z 2 ))dz 1 dz 2 = 0, 

where the contour C 2 is also a rectangle formed with four vertices ai — e±2ivoh 
and a r + e ± 2it>o/i with e > 0. One should note that the contour C2 encloses 
the contour C\. Thus, in view of (12. 74ft . it remains to find the limit of the 
following 

(2-82) - -±- 2 jf jf K\^-^)K'{^^) log(rn° fa) - m°(z 2 ))^z 2 , 
which is done in Appendix 3. 



3. The limit of mean function 



The aim in the section is to find the limit of 



J_ lKC-^)n{Em n {z)-ml{z))dz. 

ZTTl J II 

It is thus sufficient to investigate the uniform convergence nh(Em n (z) —m^z)) 
on the contour. 

Recall that F~ 1 (z) = (Em n T + 1)" 1 and then write ( see (5.2) in pQ) 

dH n (t) 



(3.1) 
where 



n(c r , 



1 + tEm r 



+ zc n E(m n (z))) = nD r> 



D n = E0i 



sjA^ 1 {z)F'\z)s 1 - -E[tr¥~\z)TA- l (z) 



n 



It follows that (see (3.20) in [T] 
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Considered z G 71 U 72 first. Applying (12. 6 p and ( 12. 5ft yields 

E (trF^ 1 (z)TA^ 1 (z)^j - E(trF~ 1 (z)TA' 1 (z)) 
= E^ 1 s T 1 A^(z)F- 1 (z)TAi 1 (z)s 1 ) 
= hE([l - 616 + 6i/3 1 e l 2 (^)]srAr 1 (^)F- 1 (^)TAr 1 (^)s 1 



(3-3) 
where 



b 1 E-trA^ 1 (z)F- 1 (z)TA^ 1 (z)T - d nl + d n2 + d n3 

n 

b l E-trA- 1 \z)F-\z)TA- l \z)T + 0{-^\ 



d nl = b\E Vl (z)(sJ A^(z)F- 1 (z)TA^ 1 (z)s 1 - -trA^(z)F' l (z)TA^ l (z)) 
i n 



d 



bi 



ii> 



E 



n 



n 



trA -1 (z)T - EtrA- 1 (z)TjsjA^ 1 (z)F- 1 (z)TA^ 1 (z)s 1 

(trA'^T - EtrA~ 1 (z)Tj 
x ^rA^ 1 (z)F- 1 (^)TA^ 1 (z)T - EtrA^ 1 {z)F- 1 {z)TA^\z)T 



E 



and 



d n3 = b x E 



f3 i e i (z)s T 1 AY 1 (z)F- 1 (z)TA^(z)s l 



1 



l3 i e i (z)( S T 1 AY 1 (z)F-\z)TA^\z)s l - -trA^(z)F~\z)TA^(z)T) 

n 

1 



+hE 

It follows from (I2.5ip and Lemma |3] that 

M 



n 



\d n j\ < 



J = 1,3, 



where we also use the fact that 
, 1 



(3.4) 



< 



n 
M 

n 



trA^ 1 (z)F- 1 (z)TA^(z)T) 



trF- l (z)F- l (z)tr(A^ 1 (z)A^ 1 (z)) 2 



1/2 M 
< 



v 



3/2 ' 



While, Lemma [3] and an estimate similar to ( I2.30P yield |G? n2 | < 



M 

^572 ' 
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Next by flZH 

nE /3iS^A- 1 (^)F" 1 (z)s 1 - E(^ 1 )E^trF- 1 (z)TA[ 1 (z)\ 
= -nb 2 1 Ek 1 s^A^ 1 (z)F- 1 (z)s 1 ] + nb\E \ /3 x ^sJ A^ 1 (z)F~ 1 (z)si 
-b 2 1 E(p i e i )E[trA^ l (z)F- 1 (z)T 

(3.5) = f n i + f n 2 + f n .3 + fni, 

where 



f nl = -nb\E r ll {s T l A- l \z)F- l {z)s l - -trA^{z)F-\z)T) 
L n 
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»2 



-b\E ^rA- 1 (z)T - EtrA- 1 (z)T S js[A^ 1 (z)F- 1 (z)s 1 
(trA-\z)T - EtrA~ l (z)T 



E 



x (trA^\z)F-\z)T - EtrA^ 1 (z)F~ 1 (z)T 



f n3 = nb 2 1 E[(3 i e i {sjA^(z)F- l (z)s 1 - hrA^F'^z)^ 



and 



U = b\[E ^i\trA- x \z)F-\z)T — E(fii£ 2 )E trA^(z)F- 1 (z)T 



We conclude from (12.411) and Lemma [3] that 

M 



Vh\f n2 \ < 



and that 



^h\fns\ < 



nv 5/2 
M 



mr 



because by ( I2.50P and Lemma 0] 



E\sTA7 1 (z)F- 1 (z)s 1 - -trA7\z)F-\z)T\ 2 < —. 

n nv 

It follows from Holder's inequality and (I2.4ip that 

\U < M(E\f3 1 \ 4 E\^\ 4 'f\E\trAY 1 (z)F- 1 (z)T - EtrAY 1 (z)F-\z)T\ 2 f 2 
M 



< 



nv 



5/2 ' 
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Therefore from (E3D, 1EE5|) . ( 1331) (J22J) and 11221)1 we obtain 

VhD n = b 2 1 E^trA^(z)F- 1 (z)TA^(z)T + f nl + 0(- ? l=) 
n n 



^£;[(T^Ar 1 ^)T^) fefc (T 1 ^Ar 1 (z)F- 1 (z)T^) fe J +0( 



fe=i 



V m; 5 / 2 



(3.6) 



-fiJ^frA^CzJP- 1 («)TAr x (z)T + 0(-4=). 



A careful inspection on the argument leading to (I2.67j) indicates that it 
also works for E-trA^ 1 (^)F- 1 (^)TA^ 1 (z)T and the main difference is that 
treating the latter does not need to distinguish between the cases j < k and 
j > k. Thus, applying (I2.67P with H = F -1 ^),^ = z 2 = z and replacing 
(j — l)/n there with one we have 
(3.7) 



—E 

n 



trTA^ 1 (z)F- 1 (z)TA^\z) 



—E 

n 



trA^ 1 (^)F- 1 (z)TFr 1 (^)T 



bi2{z)b 12 {z) 



n- 



E( K trTAi\z)TA^\z))E^rA^\z)F'\z)TF^\z)T)+0(^] 



We claim that 



(3.* 



\Em n {z)-w? n {z)\ < 



M 



nv 



3/2 



so that (12.711) is applicable. To prove ( I3.8p . we first show that 

M 



(3.9) 



\Em n {z)-nf n {z)\ < 



n v 



2 ' 



Evidently, (E2ZD yields 

\-E{trTA^{z)TA^\z)] \ < — . 

n V / v 

It follows from (12^6|) and flEZD that 



\ l -E 

n 



trTA^ (z)F (z)TA^ (z 



(3.10) 



This, together with ( 13. 6p . ensures that 



M 
< — . 

v 



(3.11) 



|D n | < 



M 
nv 
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It is proved in [6] that (see (6.38) in [6]) 



|1 - c n ml(z)Em n (z) 



t 2 dH n (t) 



> M 3 v. 



{l + tml(z))(l + tEm n (z)) 

Hence ( 13 .9p follows from the above inequality, ( 13 .lip and (13. 2p . We then con- 
clude from (fiTTP]) . (EHD and ( fTTTT]) that 

t 2 dH n (t) 



(3.12) |1 - c n ml(z)Em n (z) 



> M 2 ^, 



{l + tm°(z))(l + tEm n (z)) 1 
which, along with ( 13. lip and ( I3.2p . immediately gives ( 13. 8p . 

We are now in a position to use ( I2.7ip with z\ = z 2 = z and replacing 
(j — l)/n there with one so that 
(3.13) 

Cr, (■ t 2 dH n (t) 

Z J (l+ fm W) 2 1 

U \~..5/2^ 



-E 



n 



trTAi (z)TAi (z 



1 - c„m«(z)m»(z) / 



t 2 dH n (t) 
(l+tm°(*)) 2 



A direct application of (I2.45P and ( 13. 8 p yields 

(3.14) -E\trAT 1 (z)F- 1 (z)TF7 1 (z)T 
n L 

It follows from (l3~7D-( l3~T4l and ( 12TTUD that 
(3.15) 

V^-^ftrTA^^^F-^^TA^^^)] = v 7 /!- 



+ 0(; 



'm; 



5/2 - 



On r 
z J 



t 2 dH n (t) 
(l+im0( 2 ))3 



n 



AU±n\ z )> J (l+tm°(z)) 2 



+0( 



5/2 ^ 



We then conclude from Q, (13TB]) . (13151) . flTTO]) and (1312]) that 
(3.16) 

nh(Em (z) — mr\z)) = h ~ 



c (m°(z)) 2 f J^hML 
^n\m. n \Z)) J (1+t o f « 



+ 0( 



The case when z lies in the vertical lines on the contour can be handled similarly 
as pointed out in the last section with the truncation version of (3k{z) replaced 
with the truncation version of n(Em n (z) — m° n (z)) (one may refer to [2] as 
well). 

It remains to find the limit of the following 

c (m°(z)) 3 f t2dH " (t) 



(3.17) -r-(pK( 



,x — Zs 

IT' 



c (m°(z)) 2 f t2dHn ^ 



2 dz ■ 



which is done in Appendix 3. 
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4. The proof of Theorem [3] 



For any finite constants h, - ■ ■ ,l r ,hj Cauchy's theorem and Fubini's theorem 
we write 

(4.1) - I J K(^)dF^{y)dt) 

Ji ■ J=1 



n 



f n (t)dt-J* 3 i | K^^dF^^dt) 



n 



2hiri\ In T j=i 



oo JCi 



— )(trA -1 (,z) -ns n (z))dzdt 



n 



i — ^ 1 

K(——)dt {trA^ 1 (z) - ns n (z))dz, 
2hniJln± ~i J Ci h 

where the contour C\ is defined as before. 

Furthermore, we conclude from ( 12. 8ft and integration by parts that 



1 



— (trA _1 (z) - EtrA-\z))dz 

ft 



2hiri\/ln j- k-\ 



Ci 



K( t —^)dt\ [hg/3 k (z) 



dz 



(4.2) 



2hni 



1 -—=Y j (E k -E k „ 1 ) I K{ 



X - Z ^log$Hdz, 



h 



where in the last step one uses the fact that via ( 11. 5ft 

,x — z, 



(4.3) 



K( t —^)dt 



K( 



h 



It is observed that the unique difference between ( 14.2ft and ( 12.91) is that the 
test function there is replaced by K(^ s -). Therefore, repeating the 

arguments in Section 2 we obtain that (14.21) is asymptotically normal with 
covariance (see ( 12.821) ) 



(4.4) 



2hW\n\j Cl j C2 



if (^l^)K(^-^)log(<(z 1 )-<(z 2 ))^ 1 ^ 2 . 



h 



h 
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Also, for the nonrandom part we have 
1 



(4.5) 
Note that 



2/wiWln J J Ci 



K{ t —^)dt n(EtrA-\z) - m° n {z))dz. 



ir K{ iz_i )dtl<0O . 



Thus, repeating the arguments in Section 3 we see that (14. 5 j) becomes 



(4.6) 



4hni\/ In \ 



K(—)dt 
h 



-o(- 



r (m°(z)) 3 f t2dH "W 



rdz 



t 2 dH n (t) 
(l+tm°(z))2 



nh 3 \/\n | 



The limits of (14. 4p and (14. 6 p are derived in Appendix 3. 



5. Appendix 1 

This Appendix collects some frequently used Lemmas. 
Lemma 3. When z lies in the segments 71 U 72, 

|m°(*)| < M, \Em n (z)\ < M, ^(z)] < M, E\^(z)\ 4 < M, E\p\ r {z)\ A < M 
and 



(5.1) 



M 



—E\trA-\z)T>-EtrA-\z)V\ s < ^ 
n s n s v 12 D 



(5.2) 



.-,1 / MS M 

E\vi(z)\ 8 <—i 



n 4 v A ' 



no- 



where D is a non-random matrix with nonzero spectral norm. 

Remark 5. Lemma\^in Section 2 improves A5.1\) when \\D\\ is not bounded 
above by a constant but -trDD* < M. 

Proof. We remind readers that z = u + iv with v = Mh and u G [a, b] when z 
lies in the segments 71 U 72 . 

As pointed out in (6.1) in [6], we obtain 
(5.3) |m°(z)|<M, \m° n (z)\<M. 
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From integration by parts and Theorem 3 in [6] we have for 

/+oo 1 
d{EF A "{x) - F Cn ' Hn {x))\ 
-oo x z 

Trsup \EF An (x) - F Cn ' Hn (x)\ 



dx\ < — - < M. 



y x-zf v 
This implies 

(5.4) \Em n (z)\ < M, \Em n (z)\ < M. 
It follows from Lemma 7 and lemma 8 in |6] that 

\h(z)\<M. 

Repeating the argument of Lemma 3 in j6] gives (15.11) . 
Write 

(5.5) p*(z) = bi(z) - -/3* r (z)6 1 (z)(trA- 1 (z)T - EtrA~ x {z)T). 

n 

We then conclude that 

E\f3i r (z)\ 4 < M + —r-rEltrA-^T - EtrA^(z)T\ A 

M 

(5.6) < M+^^<M. 

rev iU 

Expand 0i(z) as 

Pi(z) = fc(z)-I3?(z)p 1 (z)m(z). 
It follows from ( I5.6p . ( 12. 7p and Lemma H] that 

1/2 



S|^ 1 (z)| 4 < J B|/3r(z)| 4 +-(i;|/3 1 (z)| 4 J B|r 7l (^f(z)| 8 



1 / \ 1/2 1 / \ 1/2 

< M + —(EMz)\ 4 m r (z)\ 4 ) < M + —(E\fk(z)\ 
Solving the inequality gives 

E\^{z)\ A <M. 
It follows from (12T25D and (JS3| that 
(5.7) \-EtrA^(z)\ < M. 



n 



By Lemma H] and (15.11) 



E\ Vl (z)\ 8 < ^-E(trA^(z)TA;\z)T)* < -E{trA- 1 \z)A- 1 \z)f 



(5. 



< 
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n 4 v 4 



E 



M (aarVw)' < M 



n 4 v 4 



n 4 v v 



^(trK7 1 {z)-EtrK7 l {z 
where A _1 (z) denotes the complex conjugate of A -1 (z) and we also use 

-EtrA7 1 (z)A7 1 (z) = -Q{-EtrA7\z)). 

n v n 

This, together with (15. ip . yields the estimate of £\(z). □ 

Lemma 4. (Lemma 2.7 of Suppose that X\, ■ ■ ■ ,X n are i.i.d real random 
variables with EX\ = and EX\ = 1. Let x = (X\, • • ■ , X n ) T and D be any 
n x n complex matrix. Then for any p > 2 

(£|X!|VDD*) p/2 + E\Xx\ 2p tr(DD*) p / 2 
6. Appendix 2 



£|x T Dx-trD| p < Mr, 



This section is to verify Remark [T] and Theorem [TJ 

We first prove that (11.101) is true when T becomes the identity matrix. When 
T is the identity matrix, the left hand of (11.101) becomes 

«(*)) a 



In view of (II .ip we have 



1 - c, 



;i+<(z)) 2 ' 



(6.1) 
and 
(6.2) 
Thus, 



1 c r . 



1 - a n 
(l + m n (z)) 2 



l--{l+zml(z)) 2 



-(z + 1 - c n ) + yj{z - 1 - c„,) 2 - 4c n 



2z 



-(z - 1 - c n ) + a/(^ - 1 - c n ) 2 - 4c,, 



1 jAc n - [Z - 1 - C n ) 



2Cri 



2c 



1 - c n )i + a/4c„ - (z - 1 - c n ) 



y/(z - 1 - C w ) 2 - 4c n 

2Cr). 



1 - C n ) + y/{z-l- C n ) 2 - 4c„ 



- 1 - C n ) 2 - 4c r , 



c n zm n {z) + z -1) 



y/(z - 1 - C n ) 2 - 4 Cn (1 + C„m»(z)) 



2r 



c n m°(z) 
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n Cn + Z-1- a/(-2 - 1 - c n ) 2 - Ac n 



where in the last two steps one uses the facts that 

(6.3) m° n (z) = 

and 

(6-4) m° n {z) = - ^ — 5^ • 

1 - c n - c n zm° n (z) - z 

It follows from (E3D and (E3D that 

(6.5) | — \ | = |1 - c n - zc n m n {z)\ < M. 

1 + c n m n (z) 

Write 

yj{z-l - c n ) 2 - 4c n = y/(a — z)(b- z). 
Then it is simple to verify that 

\y/(a-z)(b-z)\ > y/jjb - a)v. 

Thus, f ll.lOp is true when T is the identity matrix. 

Lemma 5. Under the assumptions that n 5 h 29 ^ 2 < M and that 

(6 - 6) J |i + ;<(,)!- < °°' 

U.ll\) is true and 

(6r> J R^^wr < °°' 

Remark 6. The assumptions that n 5 h 29 ^ 2 < M and \6. 6\) are only used in 
this Lemma. 

Proof. First recall that (see (6.30) in [6]) 
(6.8) \Em n (z)-ml(z)\< 11 



nv 5/2 

It is straightforward to verify that 

(6.9) |1 + tEm n (z)\ 2 - |1 + tm n (z)\ 2 < M\Em n (z) - m n (z) 

We then write 

dH n (t) f dH n {t) 



\l+tEm n (z)\ A J \l + tm n (z)\ 



|1 + tEm n (z)\ 2 - 


l + tm n (z)\ 2 dH n (t) 




l+tEm n (z) 


I 4 l 


l + tm n (z)\ 2 



(6.10) 



\l + tEm n (z)\ 2 -\ 


1 


+ tm n (z)\ 2 dH n (t) 




l + tEm n {z) 


I 2 l 


l + tm°(z)\* 
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Obviously, for the above last term, by Holder's inequality, (16. 6ft . (16. 8 j) and (16. 9 p 
\l+tEm n (z)\ 2 -\l + tm°(z)\ 2 dH n (t). 



\l + tEm n (z)\ 2 \l + tm«(z)\* 
dH n (t) 



< M\Em n {z) ~ <{z)\{ J 



dH n {t) 



(6.11) < 



M 

572 



n v 



\l + tEm n (z)\* J |l + tm0(z)|8 

dH n (t) 



1/2 



|l + t£m„. z 4 



As for another term in (I6.10p . using (16. 9p successively we have 
|1 + tEm n (z)\ 2 - |1 + tnf n {z)\ 2 dH n {t) . 



< 



< 



< 



< 



\1 + tEm n {z)\±\l + tm°(z)\ 2 1 
M\Em n {z) - nf n {z)\dH n {t) 
\l + tEm n {z)\*\l + tnt{z)\ 2 

M\Em n {z) - nf n {z)\dH n {t) [ M\Em n (z) - m°(z)\ 2 dH n (t) 
\l + tEm n (z)\ 2 \l + tm? n (z)\ A J |1 + tEm n (z)\ A \l + tmP n (z)\ A 



M 



nv 



5/2 



dH n (t) 



|l + tEm n (2:)| 4 
M M 



1/2 



< 



nv 5/2 y nr 
M 

(nv 5 / 2 ) 5 
M 



5/2)1 



+ 



+ 



+ 



M 



nv 5 / 2 ) 2 J \l + tEm n (z)\ 4 \l + tm°(z)\ 4 



M 



[nv 



5/2); 
dH n {t) 



[nv 



5/2y 



dH n {t) 



\l + tEm n {z)\ A \l + twP n {z)\ 1Q 
dH n (t) \ 1/2 



\l + tEm n (z)\* 
1/2 



1/2 



nv 5 / 2 V J \1 + tEm n (z)\ 4 



+ 



M / f dH n (t) y/2 
(nv 5 / 2 )V V 7 |l + iEw n (;z)|V ' 



where in the last step one uses (6.16) in [6]. This, together with (16. lip and 
fl6TT0|) . yields 



dH n (t) 



\l + tEmJz) 4 



<M + 



M 
572 



+ 



M 



nv 



[nv 



5/2) 



5^2 j 



dH n (t) 



\l + tEm n (z) 



1/2 



Solving the inequality gives 



dH n (t) 



< 00. 



\l + tEm n (z)\* 

Consider (JUTJ) now. By fTCT|) . (jZ7Z5jl . flBT8|) and Lemma 1 



M 



nv 



5/2 ' 
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Applying this inequality and repeating the argument for ( 11.11 j) we may prove 
(16. 7p and omit the details here. 

Proof of Theorem [TJ. Write 



nh 



,x - y s 



- j K(^)dF^(y)-f Cn>Hn (x) 



nh 



x — a 



x — b 
h 



K(y)f Cn ,H n (x - yh)dy - f Cn ,H n {%) 



By Taylor's expansion 

fc n ,H n (x ~ yh) = fc n ,H n ( x ) ~ fc n ,H n ( X )y h + fc n ,H n ( X o)(y h ) 2 ' 



where xq lies in [x — yh,x]. This, together with ( 11.7D . ( 11.151) . ( ECU]) and 
Theorem [2J ensures Theorem [TJ 



7. Appendix 3 



The aim in this section is to develop the asymptotic means and variances in 
Theorem [2] and Theorem [3] Consider (12.821) first. Note that 



2hH 2 



Ci Jc 2 



/r Ei — Z\ , X 2 — Z 2 . 
!X { —h— )K{ —h—- 



X 



In 



wSi(zi) — m^(z 2 ) + iarg(nf n (zi) - rrf n (z 2 )) dz\dz 2y 



where the contours C\ and C 2 are two rectangles defined in (I2.3P and (12.8 1 j) . 
respectively. 



As in Section 5 of j2] one may prove that 



(7.1) inf \ml(z)\ >0, 



ml(zi) -wP n {z 2 ) 



z\ - z 2 



> l\ml(zi)<(zi)\, 



where S is any bounded subset of C. 

To facilitate statements, denote the real parts of Zj by Uj,j = 1,2. In what 
follows, let n — >■ oo first and then t> — > 0. Then, as argued in [2], the integrals 
in (17.11) involving the arg term and the vertical sides approach zero. 



Define 



/r Ei - Z X X 2 - Z 2 . 



K'( Xl - Zl )K'( X2 ~ Z2 ) + K'C 1 - Zl )K'( X2 - Z2 - 
A h ' A h h } A h ■ 
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Therefore it is enough to investigate the following integrals 



h 2 7T 2 



(7.2). 



(7.3) + 



[K$ In \ml( Zl ) -rr£(z 2 )\ -K%> In |m°(^) - m^z 2 )\]d Ul du 2 



(2) 



h 2 7V 2 



a r pa r +E 



h 2 lX 2 



a r ra r +e 



h 



h 



Hkn(zi) -ZZln(^) 



du\du 2 



1( J 0.1— s 

x In 



h ' iy h ' 

ml(zi) -n£(*2))(mJK*i) -m°0 2 )) 



du\du 2 , 



where K' r {^-^) and K^(^-f^), respectively, represent the real part and imaginary 
part of K'{?^\ IRn( z ) stands for the complex conjugate of rn± n i. z )- 

We develop the limit of (17. 2p and (17.31) below. To this end, we list some 
facts below. By ( II. 5p and (II. 6p one may verify that 



(7.4) 



oo J — oo 



K'{u x )K'(u 2 ) hl(«i - U 2 ) 



du\du 2 < oo. 



In addition, it follows from ( II. 5p that 



111 h 



■r — a 
h 



x — b 
h 



Kim) 



K' r {u 2 )du\du 2 — ► 0. 



This, together with ( I7.4p . implies that as n — > oo 



1 

h 2 



x\ —a\ pX2—ai~\-£ 



x\~a r J X'i— a r — £ 



K'i^j-)K'(^-) ln(wi - u 2 ) 2 du\du 2 



h 



x\—ur I x2—ai — e 
h h 



K'( Ul )K'(u 2 ) 



ln(ui - u 2 ) 



In 



/i 2 J 



du\du 2 



(7.5) 



oo ;>+oo 



— oo «/ — oo 



K'(ui)K'(u 2 ) ln(ni — u 2 ) 2 du\du 2 



By ( II. 8p and the continuity property of K"{u + wo) and K'{u + it>o) in u and 
i>o it is not difficult to prove that 



(7.6) 

and 

(7.7) 



lim 



\K"{u + iv )\du 



lim 



+ 00 



K^\u + ivo)du 



\K"(u)\du 



K {j \u)du, j = 0,1, 



where is the j-th derivative of K. 
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By complex Roller's theorem 
lf x — Z\ 



(7. 



K 



h 



K A h + wo) 



because K^^-^) = 0, where v\ lies in (0,v )). Thus we conclude from ( 17. ip 
and (|ESj) that 



dm 



1 f b r — v 
<v hln(vZ l h)- / \K"(^— + iv 1 )\du 1 ^0, 
hj a h 

as n — >■ oo, i7 — >■ 0, which implies that f lT.3 j) converges to zero. 
Consider ( I7.2p next. We claim that for it G [ 



x—b x—a l 
h ' ft -I' 



as n — )■ oo, 



(7.9) 



|m°0„) -m(w n )| 0, 



where z n = u n — iv h with u n = x — w/i. Indeed, from (3.10) in [TJ we have 

(one may also refer to Section 6.3 of [6]). Then, as pointed out in Lemma 1 
[6], relying on this expression we may draw the conclusions for roJJ similar to 
those in Theorem 1.1 of [13] for m(z). Thus we have 



(7.11) 



|m°«) -ml(u n )\ 0. 



Also, the argument of Lemma 2 in [5] gives 
(7.12) KW-mWhO. 
Therefore, (I7.9P is true, as claimed. 
Now, as in [2], for (17.21) write 



(7.13) 



In 



rn°0i) -rrf n {z 2 ) 



m°(zi) -m2(2 2 ! 



ih.(i + 



4m° t (zi)m°.(z 2 ) 



where m"j(z) denotes the imaginary part of m° (z) . By (17. ip 



(7.14) In 1 + 



4<i(zi)mgj(^) 
|ffi°(2i) -m°(z 2 )| 2 



< In 1 



(iti -w 2 ) 2 |m°(2:i)m°(2: 2 ) 



In view of ( 17. ip and Lemma [3] 



(7.15) 



sup 

Ml ,U2 £ [a,6] ,V1 ,V2 € [«oft , 1] 



\nin(zi)m?n( Z 2)\ 2 



< OO. 
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By the generalized dominated convergence theorem we then conclude from 
(E5D, flZZZD, (ES), (EH, dH5D that asw^oo 



x l~ a l x 2~ a l ~^~ £ 

h C h 



x^—ar I &2~ a r — £ 
h h 



K' r {z x )K' r {z 2 



In 



Ekn( u ni ~ iv h) - mV(u n2 - iv h/2) 



In 



m° (u n \ - iv h) - m®(u n2 - iv h/2) 
rniuni) - m(u n2 



m(u nl ) - m(u n2 ) 



du\du 2 — > 0, 



where jh, j = 1,2. In addition, it follows from (17. 5p . (17. 7p . and 

inequalities similar to ( I7.14p and (I7.15P that as n — > oo and then v o — > 



x l — a l x 2~ a l~i~ £ 
h f h 



x\—a r I X2—o, r —£ 
h h 



(K' r ( Zl )K' r (z 2 ) - K( Ul )K' r (u 2 )) In 



m[u n i) -m[u n2 ) 



m(u nl ) - m(u n2 ) 



du\du 2 
->• 0. 



Therefore (17. 2p can be reduced to the following 

m(u nl ) - m(u n2 ) 



(7.16) 



x l~ a l x 2~ a l "l" £ 

h C h 



x-^—a r I x^—a-i — e 
h h 



K'{ Ul )K'(u 2 ) In 



m(u nl ) - m(u n2 ) 



du\du 2 + o(l), 



which turns to be 



x\—a r J X2—0,i € 



K{-)K{-)ln 



m{Xi — Ui) — m{x 2 — u 2 ) 



du\du 2 + o{l). 



m(xi — Mi) — m(x 2 — u 2 ) 

To handle (I7.16p . we need two more lemmas: 
Lemma 6. Suppose that the function g(x\,x 2 ) is continuous in x\ and x 2 , 

pxi—ai rX2-ai 

(7.17) / / \g{x\ — u%, x 2 — u 2 )\duidu 2 < oo 

J x\—a r J X2—a r 

and 

/•xi— ai fX2-ai 

(7.18) / \g{xi — U\, x 2 )\dui < oo, / \g(xi, x 2 — u 2 )\du 2 < oo. 

J X\—a T J X2— CLr 

Then, as n — > oo 

rxi-ai rx 2 -a,+e 

(7.19) — / / K'(-±)K'(-?-)g(x 1 -u 1 ,x 2 -u 2 )du 1 du 2 ^0, 

J x\—a T J X2-a r — e ft ft 

where X\ ^ a h a r and x 2 ^ a h a r . 



Define the sets G x = (|m x | < 5i) H (\u 2 \ > 5 2 ), G 2 = (|mi| > 8i) n (\u 2 \ < 5 2 ) 
and G3 = (\ui\ > 5i) fl (\u 2 \ > 5 2 ). Splitting the region of integration into the 
union of the sets (\u\\ < 5\) fl ( | ^2 1 < ^2), Gi, G 2 and G3 gives 

pxi—ai fX2—ai+e 



x\—a r J X2—<x-\ — £ 



K'(^-)K'(^-) g(x! - Mi, x 2 - u 2 ) - g(x l7 x 2 ) 



du\dui 



44 GUANGMING PAN, QI-MAN SHAO, WANG ZHOU 

(7.20) < h + h + h + h + h, 

where 



I\ = sup 

\ui\<Sl,\u2\<&2 



g(xt -ui,x 2 -u 2 ) -g(xi,x 2 ) I \K'{u)\du 



oc 



i rx 1 -a l px 2 -ai+e 

\g(x 1 ,x 2 )\ - / / I(G 1 UG 2 UG 3 )K'(^-)K'(^-)du 1 du 2 

" J x\—a T Jx2—a r —e " " 

— / )^'(-t- M^i -«i,ac 2 -u 2 )du x du 2 

J x\—a r J X2— a r -e 



and 
/ 5 = 



— / I(G 2 )K'(—)K\—)g(xi-Ui,X2-u 2 )du 1 du 2 

h Jx-i-ar Jxo-a r -£ 



x ^ r' ai+£ /(G 3 )^(^)^(^)' (xi ' Mi ' X2 " M2) ^ 2 

Xl _ ar J X2 -a r - £ h 2 h h U X U 2 



Evidently, Ji — >• due to the continuity property of g(xi,x 2 ) when <5i and <5 2 
converge to zero. As n — > oo, for J 2 we have 

/r+oo 
\K'(u)\du j \K'(u)\du^ 0, 
./ — oo 

M><5//i 

and for J 5 by ( 17. 1 8[) we obtain 

^5 < ~F~~F~ SU P \uiK'(ui)\ sup |m 2 ^'(m2)| 

0l02 |ui|><5i/ft \u 2 \>S 2 /h 
•2'i-ai rX2—ai+e 



i>x 1 -a l i-X2-ai+£ 

x / / Is^i — u ii x 2 — u 2 )\du\du 2 — > 0. 

J x\—a r J X2— a r — e 



'3:1— a r J X2— a r — £ 

Consider J 3 . Similar to Is, 

fX2-ai+e 



^ /■ rX2-ai+£ 

h < 7- sup |m 2 JT'(u 2 )| / / \K'(u 1 )g(xi-uih,x 2 -u 2 )\duidu 2 . 

"2 |u2|><52//i ./ |«i|<5i//i Jx2— a, — £ 

While, as n 00 and then 5i — > 0, by the dominated convergence theorem 



1 /" , r w/ u i< 



, \K (—)\ \{g{xx-ux,x 2 -u 2 ) - g(x 1 ,x 2 -u 2 ))\du 1 du 2 ->■ 0. 

|t/i|<<5i Jx 2 -a r ~£ 

From (17. 18[) we then see that I3 — > 0. One may similarly prove that J4 converges 
to zero as well. We summarize the above that ( 17.241) converges to zero as 
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n — > oo first and then both 8\ — > and S 2 —> 0. In addition, apparently, 

1 [ Xl ~ ai , U fh^ 

(7.21) g(xi,x 2 )- / K'(-)du = g(x 1 ,x 2 ) K'(u)du 

Jx 1 ~a r ft J h a l 

= g{x 1 ,x 2 )K(u 



h 

x-^ —a r 



h -+0. 



Thus ( 17.19ft is proved. 

The next lemma extends (1.6) in [T3], which now includes the boundary 
points of F c,H (x) under some extra conditions. 

Lemma 7. Suppose that the support of F c,H (x) is [a, b] with a > and b finite. 
Then m(x) is the unique solution to the equation 

(7.22) I= _1 +C /WW 



m(x) J 1 + Xm(x) 
where lim m(z) = m(x) . 

Proof. When u, the real part of z, is bounded, we have 

v 

^s(m(z)) > 



M + v 2 ' 
It follows that 

(7.23) - - <M + v 2 . 

Considering the imaginary parts of both sides of the equality ( 11. 3ft yields 

S(m(s)) „ . . f t 2 dH{t) 



(7-24) sup / H < ^ 



|m(2;)| 2 J |1 + tm(z) 

which, together with (17. ip and (I7.23p . implies 

t 2 dH{t) 
zes J \l + tm(z) 

Taking z — > x in (11. 3ft and using (1.5) in [T3] we then see that (17. 22ft is true. 
The uniqueness of m(x) is from continuity of m(x) and the uniqueness of m(x) 
when 3m(i) 7^ given in [T5] . □ 

We are now in a position to apply Lemma [6] to (17.161) . It follows from 
Lemma [7] that m(xi) 7^ m(x 2 ) and m(a;i) 7^ Wt{ x 2) whenever x\ 7^ x 2 . Also, 



note (5.1) in [2]. Therefore g(xi,x 2 ) = In 



mfa )—m(x?.) 



m(xi )—m(x?.) 



is continuous in X\ and 



£2- Furthermore, it is straightforward to show that In ( 1 + -, r ) 

for wi, u 2 E [ai — e,a r + e] is Lebesgue integrable and In ^1 + (^z^z^j j f° r 
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U2 G [cti — e, a r + e] is Lebesgue integrable. Thus, in view of inequalities similar 
to fl7.13p - fl7.15p and applying (I7.19P we have 
(7.25) 



1 



x i~ a i rx2—ai+£ 



K'(^)K'(^)\n 



x\—a r J X2— a r — e 



h 



h 



m(xi — u\ ) — m[X2 — u 2 ) 



m\x\ — U\ ) — m(x 2 — U2 ) 



du\du<i — > 0, 



which is the limit of (17.21) due to (I7.16P when x± ^ x 2 . 



When x\ = x 2 = x taking g(xi, x 2 ) = In m(x) — m(x) and applying (I7.19P 
we obtain 



(7 ' 26 > s? /. 



x — a\ rx—ai-\-e 



x—a r J x—a r —e 



A-'(^)A-'(^)l„ 



m{x—Ui)—m{x—U2 



du\du2 — > 0. 



Here we keep in mind that the boundary points are not considered when in- 
vestigating the case X\ = x 2 = x. Consider next 



(7.27) ^ 



'x-b h 

By complex Roller's theorem we have 

In m{x — U\ ) — m[x — U2, 



U2 

K'(—) In mix — Ui) — m(x — u 2 ) 

x—b—e " 



du\du 2 



= ~ In (ju! - u 2 ) 2 [\m! r (x - m 3 )| 2 + |m-(x - m 4 )| 2 ] ) 
(7.28) = -\n(u 1 -u 2 ) 2 + -g r i(x -u 1: x -u 2 ), 

where g r (x — ui,x — u 2 ) = In (\rri! r {t\(x — U\) + (1 — ti)(x — U2))\ 2 + ImK^Oe — 

■ui) + (1 - t 2 )(x - m 2 ))| 2 ), m 3 = txUi + (1 - ti)u 2 , U4 = t 2 u\ + (1 - t 2 )u 2 and 
ti,t 2 G (0, 1). It follows from inequalities for m(x) similar to (17. ip that 



x— a rx—a+e 



hi 



m[x — ui) — m[x — u% 



du\du 2 



1 x—b J x—b—e 

This, together with (I7.28p . ensures that 

fx—a px—a+e 

/ g r (x — iti, x — U2)du\du2 

x—b J x—b—e 



< OO. 



< OO. 



Similarly, one may verify the remaining conditions in Lemma EJ Therefore, 
using Lemma [6] with g(x\,x 2 ) = In |m'(x)| 2 gives 



(7.29) ^ 



x—b 



A "<¥> 



x—b—e 



K'{-^-)g r (x — Mi, x — u 2 )duidu 2 — > 0. 
h 
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We then conclude from (E2HD, (17391) and (JZ5} that 
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(E2ZD 



l l 

2h? 



x—b 

+00 /"+00 



x—b—e 



K'(—) ln(«x — u 2 ) 2 du\du 2 + o(l) 



(7.30) "^o/ / i^'(wi)^'(w2)ln(«i - u 2 ) 2 d Ul du 2 . 

^ J —oo J— oo 

which is the opposite number of the limit of (I7.2p due to (17.261) and (I7.16P 
when x\ = x 2 . 

Limit of ( 13.171) . From an expression similar to (11.31) we obtain 



It follows that (I3.17P becomes 
1 



{rrf n {z)f 



(7.31) 



^(^)-f log 

h dz 



Aixhi 



1-c 



t 2 «(z)f 
l+tm? n (z)) 2 



dH n (t) dz 
dH n (t) 



dz 



l+tw^{z)) 2 
1) in [I] t 

:i + t < ( z)T dH ' m> - Mv - 

This implies that the integrals on the two vertical lines in (I7.3ip are bounded 
by Mvlogf -1 , which converges to zero as v — > 0. The integrals on the two 
horizontal lines are equal to 



As in the inequality above (6.37) in [6] and (3.21) in [T] one may prove that 
(7.32) |l- c f «' 



(7.33) 
(7.34) 



1 



+ 



2nh 
1 

2nh 



K' 



h ' 

1 h ' 



l + tml(z)) 2 
t 2 {w? n {z)) 2 



arg 



1-c 



dH n (t) 
T dH n (t) 



du 
du. 



By (2.19) in [2], (EH} and ( Q2} we see that (1735]) is bounded by Mwlogv 
converging to zero. It follows from ( 17. 9p and Lemma [5] that 



-i 



* 2 W(^)) ; 



;i+i<(*n)) 2 

We also claim that 
(7.35) ' t2( ^ (M " ))2 



<2#„(t) 



'1 + tm(u n )) 



-dH n {t) 



t 2 (m(u n )) 2 
(1 + tm(u n )) 2 

t 2 (m(u n )) 2 
(1 + tm(u n )) 



dH n (t) 0. 



r dif(*) ->■ 0. 
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To see this, introduce random variables T n having distribution H n (t) and T 

having distribution H(t). Then T n T. Also T n and T are both bounded. 
Consequently by Lemma [5] 



E 



T 2 



T 2 



1 + T n m{u n )) 2 (1 + Tm(u n )y 



< E 



T 



T 



E\ 



T 



+ 



T 



1 + T n m(u n ) 1 + Tm(u n ) 1 + T n m(u n ) 1 + Tm(u n ) 



1/2 



<M IE 



1 



>E\ 



y E\T„, - T\ 



1/3 



1 + T n m{u n ) 1 1 1 + Tm(u ri 
converging to zero. Thus (I7.35P is true, as claimed. We then conclude from 
the dominated convergence theorem that 

t 2 (m°(z n )) 2 



K' r (z) arg 



1-c 



— arg 



1-c 



l + tmP n (z n )f 
t\m{u n )) 2 ■ 



Moreover, by ( I7.7|) we obtain 
(K' r {z)-K' r {u))axg 



l + tm(u n )) 2 

t 2 (m(u n )) 2 



1-c 



du ->■ 0. 



-dH(t) 



du 0. 



(l + tm(u n )) 2 

By (17.2 ip and Theorem 1A in [TQ] (replacing K{x) there by K'(x)) we see that 

t 2 (m{u n )) 2 



K' r {u)) arg 



1-c 



r di7(t) 



du -> 0. 



;i+tm(« n )) 2 

Summarizing the above yields that (I3.17P converges to zero. 

Limits of ( 14.4ft and ( 14. 6j) . Repeating the argument leading to (I7.16P yields 



becomes 



(7.36) 
1 



h 2 \n- , 

h J x\—a r J x'2 — d r — e 



m(xi — Ui) — m(x 2 — u 2 ) 



duidu 2 +o(l) . 



m(x\ — ui) — m(x 2 — u 2 

The argument of (I7.24p in Lemma [6] indeed also, together with (11.61) . gives 
(7.37) 
1 

h 2 



K(-^)K(^)g(xi - ui, x 2 - u 2 )du\du 2 - g(xi, x 2 ) -> 0. 



x\— a r J X2— a r — e 

This ensures that (I7.36P converges to zero when x\ ^ x 2 . When x\ = x 2 = x, 
by (I7.37P we have 

I rxi-ai rX2-ai+e u ^ u ^ 

— — -/ / K(—)K(—)\nm(x 1 -Ui)-M(x 2 -u 2 )du 1 du 2 ^0. 

"> ^ n fl J x\-a r Jx2-a r -e 
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Applying l|737j) and replacing K'(x) in (17^)1 . (17251) . (1725)1 and ( 17301) by K(x), 
we can prove that 



x l~ a l /•Z2—0.1+E 



h 2 \n- I 



kC^-)K(-^) In m(xi— Ui)-m(x 2 -M 2 ) 
ft, ft 



du\dui 1. 



Checking on the argument of flSTTTJ and replacing K'(x) there with K(x), 
along with (14 .3p . we have 

gSD -> 0. 
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